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13 April 1977 


MEMORANDUM FOR: Deputy Director for Administration 
Deputy Director for Intelligence 
Deputy Director for Operations 
Deputy Director for Science $ Technology 
Executive Secretary 

THROUGH : Comptroller 


SUBJECT 


Resources for Program Evaluation in Fiscal Year 1977 
-- GMB Bulletin 77-8 


1. A report showing and commenting on the CIA resources devoted 
to Program Evaluation is due in the Office of Management and Budget on 

_25_ April. It is in our interest that the CIA submission show as 
thoroughly and carefully as possible Agency efforts to systematically 
analyze its programs to ascertain the extent that they "have achieved or 
are achieving their objectives, or are producing other significant 
effects...." 

2. A copy of the actual GMB Bulletin is attached; it requires 
some interpretation. Basically the data desired is that showing the 
extent that CIA attempts to determine "what has actually occurred as the 
result of past program actions," plus ‘our efforts to develop evaluation 
methodologies and ensure that these are incorporated into existing and 
future programs. The efforts identified should be intended to assist 
management. 

3. For the purposes of this GMB tasking, program evaluation 
does not include: 

-- evaluation activities .which are an intrinsic part of pro- 
gram operations or management; 

-- analyses of the relative cost/benefit of alternative ' 
approaches to programs that do not yet exist; 

-- the development and operation of management information 
systems ; 

-- the collection of routine data or the preparation of general 
statistics or basic research. 
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4. Under such criteria,. we would assume that, for example, 
approaches to the development of a new technical collection system would 
not be included, but that th^evaluation of performance of existing 
systems would be. Special^^^B efforts to monitor the effectiveness of 
particular products would D^mcluded when these were expected to be 
used as a basis for new policy and management decisions. Formal studies 
of' the DDO Evaluation and Program Design Staff and others of collection 
projects would be included. The efforts of much of the DDI Center for 
Policy Support is aimed at determining the utility of finished intelli- 
gence. Some of the work of the DDA Center for the Study of Intelligence 
might be included. A portion of the efforts of the Inspections Staff 
and the Audit Staff deal with program evaluation. The NIOs engage in 
program evaluation when they participate in "FOCUS- type" studies or 
consider the performance of collectors relative to Key Intelligence 
Questions. 


5. The attachment to the (MB Bulletin shows the prescribed format 
for your input. Please give us a name of a contact person; we will not 
pass that name to CMB, We propose to provide CMB with data for each of 
the four Directorates and for the Office of the Director. One staff-year 
should be counted for each full-time person who devotes more than fifty 
percent of his/her time to program evaluation activities or to monitoring 
such activities by contractors ; part-time employees or employees working 
on evaluations, less than fifty percent of their tim<p are not to be 
included. A brief narrative description of program evaluation activities 
and of their expected impact is to be included (two or three paragraphs 
per Directorate). Significant increases or decreases in funding levels 
for evaluation activities from FY 1977 to FY 1978' are to be noted and 
major "policy or strategy changes" mentioned. 

6. A two- three page .Agency summary narrative will be prepared by 
this office and we welcome your suggestions. The data you provide 
should include activities of your components in support of interagency 
evaluation efforts (e.g., by the IC Staff) or of evaluation activities ' 
of other agencies.. The Agency overview will note that CIA regularly 
participates in evaluations of Programs under the control of other 
Departments and Agencies . 

7. Your data and narrative are needed by no later than COB 

21 April . We apologize for the tight deadline. But the OMB tasking only 
arrived in the Com ptroller's Offi ce on 12 April, and the OMB deadline is 
Monday, 25 April. of my Staff (Red 1829, Rm 3E58) , is 

coordinating this eftor^incHi^temains ready to talk further with your 
people regarding the nature and extent of the task. 

25X1 A 


Assistant Coup t roller 
Requirements 8 Evaluation 

At t admen t : 
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WASHINGTON, D.C. 20503 




BULLETIN NO. 77-8 


Aoril 4, 1977 


TO THE HEADS OF EXECUTIVE DEPARTMENTS. AND ESTABLISHMENTS 
SUBJECT: Resources for Program Evaluation in Fiscal Year 


1. Purpose . This Bulletin provides instructions for the 
submission of information concerning the resources for 
program evalu ation pr ovid e d for i n fiscal year ~T9 7 7 . 

Background . The Office of Management and Budget has 
determined the need to collect and analyze information on 
the level of Federal expenditures and staffing devoted to 
program evaluation activities. The information requested by 
this Bulletin is vital to an understanding of the magnitude 
of evaluation resources and the ways in which • these 
resources contribute to the achievement of Rational goals, 
agency missions, and program objectives. Moreover, 
increasing concern about the adequacy of data on program 
performance make.s it necessary to assure that accurate 
information can be provided to the Congress and the public 
on the actual level of program evaluation expenditures. 

3. Coverage . These instructions apply to each executive 
department and independent agency with more than 800 
authorized full-time permanent employees. 

4 * Definition . For purposes of this Bulletin, program 
evaluation means the formal assessment, through objective 
measurements and systematic analyses, of the manner and 
extent to which Federal programs (or their components) have 
achieved or are achieving their objectives, or are producing 
other significant effects of either a desirable or 
undesirable character, in order to assist future policy and 
management- decisions. 

a. The following will be considered as Federal program 
evaluation activities for the pu rpose of this Bulletin: 
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(1) Formal studies, surveys, and data analyses for 
the purpose of determining the effectiveness, efficiency, or 
impact of a national or regional program. 

(2) Systematic assessment of demonstration programs 
or projects which are expected to have major implication for 
programs of national or regional scope; except that 
evaluation- activities which are an intrinsic part of the 
program operations or management should not be reported. 


(3) Formal studies, surveys, and data analyses .-for 
the purpose of assessing program management processes, 
procedures, intervention techniques, workload distribution, 
or productivity. 

. (4) Assessment and development of program designs 
to assure that programs, once operational, can be 
successfully evaluated. 


(5) Design, development, and field testing of new iS, 
program evaluation methodologies. 

(6) Synthesis and further analysis of results 
obtained by several previous program evaluation efforts. 


(7) Collection of initial data to help in 
evaluation design and provide a baseline for subsequent 
evaluations. 

bo The following will • not be considered as Drogram 
evaluation activities for the purposes of this Bulletin. 

(1) Design, development, and operation of general' 
data systems or management information systems. 

(2) Continuing collection of routine data and 
general purpose statistics. 

(3) Analysis of existing or proposed policies where 
no programs yet exist {even though authorised) for purposes 
of appraising the likely costs and effects of feasible 
alternatives. Although such analyses are often called 
"evaluations," they are prospective in character? whereas 
program evaluation is retrospective, aimed at determining 
what has actually occurred as the result of past program 
actions. 


(4) Basic research and studies intended to increase 
or foster general knowledge development, but which are not 
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expected to 5e used specifically and primarily in policy and 
management decisions. 


(5) Routine, day-to-day monitoring of program 
operations which is an intrinsic part of program 
administration. 


c * It is recognised that the present extensive 
variation among Federal agencies and programs will require 
case-by-case judgments to determine the applicability of the 
above criteria to agency "evaluation" activities. 

**• Reporting Requirements . Departments and agencies should 
submit a report in accordance with the attached instructions 
ana exhibit to OMB, attention: Evaluation and Program 
Implementation Division by April 25, 1977. This is a one- 
jtime report . ' 

6. Rescission . This Bulletin is rescinded following 
receipt of the April 25, 1977 reoort. 

7 • .Inquiries . Inquiries and requests for assistance should 
be directed to the Evaluation and Program Implementation 
Division, Office of Management and Budget. (395-4726 or 
395-^642. ) 



Bert Lance' 
Director 


Attachment 
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INSTRUCTIONS 


An original and one copy of the report on program evaluation 
resources will be prepared on letter size sheets (8x10-1/2) 
end submitted in the format of the attached exhibit, as 
prescribed below. ' 

Date ~ Enter the date of preparation 

C ontact - Enter the name and telephone number of an 
individual who can respond to questions relative to 
the information provided. 

Part I. RESOURCES SCHEDULE: 


Pepartment/Agencv/Bureau - Enter the 
independent agency. In the case of 
enter each bureau or comparable- 
which separate evaluation resources 
can be identified, a consolidated 1 
organizational units (if applicable), 
the total department or agency. At 
be left between each line entry. 


reporting department or 
cabinet departments, 
organizational unit for 
(funds and personnel) 
ine entry for any other 
and a line entry for 
least one space should 


iL£g- £f . year s “ Enter the estimated staff-years for full-time 
permanent personnel who devote more than fifty percent of 
their time to the program evaluation activities outlined in 
paragraph 4a of this Bulletin. _ This includes full-time 
government personnel engaged 'in monitoring or directing such 
activities by contractors or grantees. 


“ Enter the estimated obligations for personhel 
compensation related to the staff-year entry. 


Personnel Benefits - Enter the estimated obligations for 
personnel benefits associated with the salary figure. 

Co ntracts/Grants — Enter the estimated obligations to be 
incurred for evaluation contracts or grants, or, in the case 
of multi-purpose agreements, the obligations for those 
portions of the contracts or grants that are directly 
related, to the program evaluation activities described in 
paragraph 4a of this Bulletin. 


P^ r : er - E xpenses - Enter the total obligations 
Oi_ner administrative expenses directly 
performance, commissioning, or oversight of 


estimated for 
related to the 
the activities 
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described in paragraph 4a of this Bulletin. "Other 
expenses" would include obligations for travel and 
transportation of persons, transportation of things, rent, 
communications and utilities, printing and reproduction, 
other ^ services (such as ADP services), supplies and 
materials, and equipment. Significant obligations for any 
of these "other expenses" should be briefly identified in 
the narrative. 

Total ~ Enter the total estimated FY 1977 obligations for 
salaries, personnel benefits, contracts/grants, and other 
expenses. Note: All obligations are to be entered in 

thousands of dollars. 

Part II. NARRATIVE STATEMENT . 


A brief explanatory statement should describe in general 
terms the program evaluation activities being undertaken and 
the objectives toward which they are directed. It should 

highlight significant increases or _de creases in funding 

le Y el s ■ from t h e__ _ n cu r reB jEZ,... JXscaX. .veazb*t q _F Y l978'‘“and'‘nTa‘jor 

jSoli cy or s t r a t egy,^..changes-^axpec ted_„.t o,.,„.,occu r ^"'"’‘Efrtphhsir s 
snould Tbe”' placed on the expected use “of evaluation 
information in the agency resource allocation and other 
decision processes. In most instances, t‘he narrative for 
individual bureaus should be restricted to two or three 
paragraphs and the departmental or agency summary should r.ot 
exceed two pages. 
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EXHIBIT 
Bulletin No 


REPORT ON FY 1*77 PROGRAM EVALUATION RESOURCES 
(Dollars in Thousands) 


Date: 
Contact : 


Part I. RESOURCE SCHEDULE 


Staff Years- Salaries 

D epartmen t/ Age n cy/B ureau (FTP) ■ ($) 

Department of Government 
(Report breakdown of major 
bureau or other organiza- 
tional unit for cabinet 
departments only) 


Personnel 
Benefits 
(?) 


Contracts 

Grants 

(?) 


Other 

Expenses 

(?) 


PART II. NARRATIVE STATEMENT 
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( 


TOTAL 

(?) 


< 
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Existing ODP Program Evaluation Methods 

Management by Objectives Program 
Period - Bi-Monthly and Annually 

Scope - Projects of Agency, Directorate of Office 
importance which have Action Plan 

"Board of Directors" 

Period - Monthly and on Demand 

Scope - Anthing of interest to Director, initially 
follows MBO 

Program Call and Revision to Program Call 
Period - Annually and Cyclic 

Scope - Only that portion devoted to evaluation 
Project Continuance Review 

Period - Monthly (all projects once a year) 

Scope - All prodution systems for which ODP is 
responsible 

Project Managem^nj 

Period - According to system development schedule 

Scope - Review and evaluation of past activity and 
future plans, according to project schedule. Examples 
are feasibility study evaluation, preliminary design 
review, critical design review and acceptance tests, et. 

Hardware/Software Procurement Concurrence 

Various Ad Hoc Evaluation Groups 

Steering Committees 

GIMS Project Reviews 

Configuration Management 

Project Team Evaluation and reviews 
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Data Systems Which Aid in Program Evaluation 
Project Activity Report 
Period - Monthly 

Scope - All projects which use ODP resources 
Applications Tracking System 

Period - On-line, Weekly and Monthly Reports 

Scope - All projects which use Applications resources, 
and/or for which Applications is responsible, tracks 
progress. 

SMF Accounting 

Period - Daily, Monthly, Annually 

Scope - Support all machine resource accounting 
systems 


Approved For Release 2001/05/23 : CIA-RDP83T00573R000600030006-0 



